Conventionally, streaming of video and audio via the Internet and so forth has been performed. At this time, in the event that packet loss or arrival delay occurs, there is the concern that data quality deterioration may occur. For example, in the case of an encoding method which takes inter-frame difference, such as with MPEG (Moving Picture Experts Group) or the H.26x family compression methods, if there is dropout of data of a certain frame due to packet loss, so-called error propagation, which affects the image quality of subsequent frames, occurs.
Also, with the MPEG method, the compression rate is raised by motion estimation, but performing motion estimation makes algorithms complicated, and the processing time increases proportionately to the square of the frame size, so encoding delay of several frames occurs in principle. In the event of performing bi-directional real-time communication, the delay time is very close to the allowed delay time of 250 ms, and is of an unignorable magnitude.
In contrast to these, intra-frame codecs such as represented by JPEG (Joint Photographic Experts Group) 2000 do not use inter-frame difference information, so delay such as described above does not occur. However, compression is performed in increments of frames, so there is the need to wait at least 1 frame until starting of encoding. There are 30 frames per second with common systems nowadays, so a wait time of around 16 ms is needed to starting of encoding.
There has been demand for further reducing of this delay, and reduction in delay at portions other than encoding and decoding has also become necessary.
As for one of processing delay, there is packetizing/depacketizing processing wherein RTP (Real-time Transport Protocol) packetizing and depacketizing is performed. Conventionally, at the time of performing depacketizing, processing is started after waiting for a certain amount of packets to accumulate, and accordingly delay has occurred. As for a reason for accumulating packets, there is a limit on the maximum size of a packet which can be transmitted over the Internet without being divided, and a series of data having a certain meaning is transmitted having been divided into multiple packets. At depacketizing as well, buffering is performed until the divided packets are all present, following which depacketizing is started, so delay increases, and also resources for the buffer are necessary.
That is to say, with the device at the reception side of encoded data, received packets are depacketized by a depacketizing processing unit, and extracted encoded data is decoded by a decoder, but at this time, there is the need to have a buffer at both the depacketizing processing and decoding processing, so there was the concern that the buffer memory capacity might increase.
On the other hand, various methods have been conceived for reducing the buffer memory capacity (e.g., Patent Document 1).
In Patent Document 1, at the same time as packets received by a relay device being accumulated in buffer memory for error correction, sequential transmission toward the downstream side is performed without waiting for the packets of the entire error correction computation block to arrive, and in the event that a missing packet is detected, this is restored by performing error correction computation, and the recovered packet is transmitted in a format following the packets sent ahead.    Patent Document 1: Japanese Unexamined Patent Application Publication No. 2005-12753